Efficient Reliable Multicast on Myrinet

Although multicast is an important communication primitive for parallel programming, many modern networks do not support it in hardware. Multicast can be implemented in software on such networks, using some spanning tree protocol. Making multicast reliable, however, is a difficult problem, even if the hardware point-to-point communication is reliable. The key issue is that a flow control mechanism is needed to prevent overflow of software buffers. Without flow control, messages may have to be dropped, resulting in unreliable communication. Flow control for multicast communication is hard, because buffer space at many processors is involved. This paper describes a reliable multicast algorithm, using a flow control method based on a credit scheme. It also describes the implementation of the algorithm on Myrinet, which supports reliable point-to-point communication but no multicast. Our multicast algorithm has been implemented by extending the Illinois Fast Messages software. To obtain low latency and high throughput, forwarding of messages is handled entirely by the network adapter boards without involving the hosts. Measurements on an 8-node Myrinet system show that the implementation achieves high performance.